15 results
Contents
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp v-vi
-
- Chapter
- Export citation
3 - Understanding Darkweb Malicious Hacker Forums
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 13-37
-
- Chapter
- Export citation
-
Summary
Introduction
For companies and institutions of all kinds, matters regarding the protection of Intellectual Property (IP) and Personally Identifiable Information (PII) from cyber-breaches and data-leaks are demanding higher financial investment. With the discovery of Stuxnet, offensive and defensive cyber-capabilities have become a tool in military arsenals worldwide and are on the cusp of shifting the global landscape of military power. With the expanding yield of cyber-related activities, understanding the actors creating, manipulating, and distributing malicious code becomes a paramount necessity.
After discussing the commercial importance of cyber threat intelligence in Chapter 2, we will begin learning how these cyber threat intelligence systems are built. The first logical step, which will be covered in this chapter, is to introduce the online hacker communities from which so much cyber threat intelligence derives. In this chapter, we report on the results of an exploration of black hat hacker forums on both the Internet and crypto-networks (in particular those accessed via the Tor-browser). We report on the structure, content, and standards of behavior within these forums. Throughout, we highlight how these communities augment the activities of the malicious hackers who participate.
Some of the English-language forums we will discuss are accessible though the Tor-network only, while the web forums addressing Russian speakers are most often found on the surface-layer Internet. These arenas of communication between malicious hackers allow insights into concerns, motivations, and goals as well as the environment in which they act. An intimate understanding of these communities will greatly aid proactive cybersecurity [9], by allowing cybersecurity practitioners to better understand their adversaries.While the structure of these forums largely resembles similar platforms, it is in the content and members that they differ.
Valuable insight into the structure and culture of hacker communities can be gained by focusing on forums where hacking techniques and exploits are created, shared [104], and distributed [23, 41]. Furthermore, these platforms often enforce rules of conduct, discuss the legitimacy of future endeavors, and negotiate targets [51, 9]. As such, forums constitute arenas in which the propagation of hacking techniques as well as discussion on cracking and ethics take place [41, 25]. Concerns, ambitions, and modi operandi of malicious hackers are showcased in forums, suggesting that a profound understanding of these communities will aid in early detection of cyber-attacks. The study in this chapter represents initial research in this direction.
8 - Conclusion
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 118-122
-
- Chapter
- Export citation
-
Summary
Introduction
In this chapter, we describe the unique challenges to the important problem of sociocultural modeling of cyber threat actors and why they necessitate further advances in artificial intelligence—particularly with regard to interdisciplinary efforts with the social sciences.
Cybersecurity is often referred to as “offense dominant” alluding to the notion that the domain generally favors the attacker [67]. The reasoning behind this is simple: a successful defense requires total control over all pathways to a system while a successful attack requires only one. As a result, any given cyber-defense based on the hardening of systems will fall prey to a cyberattack as perpetrators gain knowledge and resources. Solutions have ranged from sophisticated adaptive defense strategies to offensive cyber-operations directed against malicious hackers. However, these methods have various technical shortcomings—which range from the technical immaturity of adaptive defenses to consequences of aggressive cyber-counteroperations. This process can lead to undesirable effects such as preemptive and preventative cyber war.
More and more, the cybersecurity industry has been moving toward the threat intelligence that we have been highlighting throughout the book, with the end goal being to preempt cyber-attacks before they occur. Discussed thoroughly in Chapter 3, a key source of cyber threat intelligence lies in the digital communities of malicious hackers—consisting of sites, markets, chat-rooms, and social media channels where information is shared, hackers are recruited, and the latest malware and exploits are bought and sold. Artificial intelligence and machine-learning techniques for analyzing communities on the Internet are long-established across specialty areas such as data-mining, information retrieval, and web science. However, we argue that the study of hacker communities combined with the goal of automating the collection and analysis of information about the activity of cyber threat actors, produces some very unique challenges. In this chapter, we describe some unique characteristics of cyber threat sociocultural environments and several challenging modeling problems for which various artificial intelligence techniques can be used to help solve.
Environmental Characteristics
When introducing hacker communities in Chapter 3, we studied them from a qualitative standpoint. We noted several unique characteristics in the online sociocultural environments frequented by malicious hackers that make these communities distinct from other groups. Some of these characteristics include the following.
Preface
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp ix-x
-
- Chapter
- Export citation
-
Summary
Rapidly emerging is an exciting new field known as “cyber threat intelligence.” The key idea with this paradigm is that defenders of computer networks gain a better understanding of their adversaries by analyzing what assets they have available for an attack. In this book, we examine a new type of cyber threat intelligence that takes one into the heart of the malicious hacking underworld— the darkweb. These highly secure sites have allowed for an anonymous community of malicious hackers to exchange ideas, techniques, and buy/sell malware and exploits. This book examines how we explored this problem through a combination of human and automated techniques to grasp a better understanding of this community. We describe both methodology and some of the resulting insights. This book serves as a first step toward a better understanding of malicious hacking communities on the darkweb.
The authors would like to acknowledge the generous support from the Arizona State University Global Security Initiative (GSI), the Office of Naval Research Neptune program, the Arizona State University Institute for Social Science Research (ISSR), and CNPq-Brazil, which have enabled our research in the area of cyber threat intelligence mined from the darkweb. Specific individuals, we would like to thank include Jamie Winterton, Nadya Bliss, H. Russel Bernard, William Brandt, Andrew Gunn, Robert Morgus, Frank Grimmelmann, Amanda Thart, andVineet Mishra.We alsowould like to extend a special thanks to Lauren Cowels, our editor at Cambridge University Press, whose assistance throughout the creation of this book was much appreciated.
Frontmatter
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp i-iv
-
- Chapter
- Export citation
6 - Using Game Theory for Threat Intelligence
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 67-95
-
- Chapter
- Export citation
-
Summary
Introduction
Penetration testing is regarded as the gold-standard for understanding how well an organization can withstand sophisticated cyber-attacks. In a penetration test, a “red team” is hired to expose major flaws in the firm's security infrastructure. Recently, however, the market for exploit kits has continued to evolve and what was once a rather hard-to-penetrate and exclusive market—whose buyers were primarily western governments [95], has now become more accessible to a much wider population. In particular, 2015 saw the introduction of darknet markets specializing in zero-day exploit kits—exploits designed to leverage previously undiscovered vulnerabilities. These markets, which were discussed in Chapters 3–5, make exploits widely available to potential attackers. These exploit kits are difficult and time consuming to develop—and are often sold at premium prices. The cost associated with these sophisticated kits generally precludes penetration testers from simply obtaining such exploits, meaning an alternative approach is needed to understand what exploits an attacker will most likely purchase and how to defend against them. In this chapter, we introduce a data-driven security game framework to model an attacker and a defender of a specific system, providing system-specific policy recommendations to the defender. In addition to providing a formal framework and algorithms to develop strategies, we present experimental results from applying our framework, for various system configurations, on a subset of the real-world exploit data gathered from the system presented in Chapter 4. This game theoretic framework provides another example of rich cyber threat intelligence that can be derived from the darknet exploit data.
For this chapter, we surveyed 8 unique marketplaces and show some example exploit kits from the data set in Table 6.1. The widespread availability of zero-day exploits represents a potential game changer for penetration testers— specifically posing the following questions:
• What exploits will an attacker likely purchase if he targets my organization?
• What software used in the organization pose the biggest risk to new threats?
To address these challenging questions, we extend a data-driven security game framework, initially introduced in [90]. Given a system configuration (or a distribution of system configurations within an organization) we model an attacker who, given a budget, will purchase exploits to maximize his level of access to the target system. Likewise, a defender will look to adjust system configurations in an effort to minimize the effectiveness of an attacker while ensuring that necessary software dependencies are satisfied.
References
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 127-134
-
- Chapter
- Export citation
4 - Automatic Mining of Cyber Intelligence from the Darkweb
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 38-55
-
- Chapter
- Export citation
-
Summary
Introduction
Now that we have a better understanding of the hacker communities present on both the darknet and the clearnet, which were discussed in the previous chapter, we can begin to use data-mining and machine-learning techniques to aggregate and analyze the data from these communities, with a goal of providing valuable cyber threat intelligence. This chapter is an extension of the work in [80]. We present a system for cyber threat intelligence gathering, built on top of the data from communities similar to those presented in Chapter 3. At the time of writing, this system collects, on average, 305 high-quality cyber threat warnings each week. These threat warnings contain information regarding malware and exploits, many of which are newly developed and have not yet been deployed in a cyber-attack. This information can be particularly useful for cyberdefenders. Significantly augmented through the use of various data-mining and machine-learning techniques, this system is able to recall 92% of products in marketplaces and 80% of discussions on forums relating to malicious hacking, as labeled by a security analyst, with high precision. Additionally, we will present a model based on topic modeling used for automatic identification of new hacker forums and exploit marketplaces for data collection.
In succeeding sections, we will introduce a machine-learning-based scraping infrastructure to gather such intelligence from these online communities. We will also discuss the challenges associated with constructing such a system and how we addressed them. Figure 4.1 shows the number of detected threats for five weeks and Table 4.1 shows the database statistics at the time of writing, which indicates that only a small fraction of the data collected is hacking related. The vendor and user statistics cited only consider those individuals associated in the discussion or sale of malicious hacking-related material, as identified by the system.
Specific contributions of this chapter include:
• Description of a system for cyber threat intelligence gathering from various social platforms from the Internet such as deepnet and darknet websites.
• The implementation and evaluation of learning models to separate relevant information from noise in the data collected from these online platforms.
• A machine-learning approach to aid security experts in the discovery of new relevant deepnet and darknet websites of interest using topic modeling—this reduces the time and cost associated with identifying new deepnet and darknet sites.
1 - Introduction
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 1-3
-
- Chapter
- Export citation
-
Summary
Recently, the online market for exploit kits, malware, botnet rentals, tutorials, and other hacking products has continued to evolve, and what was once a rather hard-to-penetrate and exclusive market—whose buyers were primarily western governments [95]—has now become more accessible to a much wider population. Specifically, the darknet—portions of the Internet accessible through anonymization protocols such as Tor and i2p—has become populated with a variety of markets specializing in such products [94, 2]. In particular, 2015 saw the introduction of darknet markets specializing in zero-day exploit kits, designed to leverage previously undiscovered vulnerabilities. These exploit kits are difficult and time consuming to develop—and often are sold at premium prices.
The explosive increase in popularity of exploit markets and hacker forums presents a valuable opportunity to cyber defenders. These online communities provide a new source of information about potential adversaries, consequently forming the nascent cyber threat intelligence industry. Pre-reconnaissance cyber threat intelligence refers to information gathered prior to a malicious actor interactingwith a defended computer system. To provide a concrete example demonstrating the importance of pre-reconnaissance cyber threat intelligence, consider the case study shown in Table 1.1. A Microsoft Windows vulnerability was identified in February 2015. Microsoft's public press release regarding this vulnerability was essentially their way of warning customers of a security flaw. At the time of its release, there was no publicly known method to leverage this flaw in a cyber-attack (i.e., an available exploit). However, about a month later, an exploit was found to be on sale in a darknet exploit marketplace. It was not until July when FireEye, a major cybersecurity firm, identified that the Dyre Banking Trojan, designed to steal credit card information, exploited this particular vulnerability. This vignette illustrates how threat warnings gathered from the darknet can provide valuable information for security professionals in the form of early-warning threat indicators. Between Dyre and the similar Dridex banking trojan, nearly 6 out of every 10 global organizations were affected, a shocking statistic.
Index
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 135-137
-
- Chapter
- Export citation
5 - Analyzing Products and Vendors in Malicious Hacking Markets
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 56-66
-
- Chapter
- Export citation
-
Summary
Introduction
Chapter 3 introduced darknet hacker communities and marketplaces, with Chapter 4 presenting a system for gathering data from these sites. In this chapter, we extend the work from [70], presenting techniques to analyze the aggregated dataset, with a goal of providing rich cyber threat intelligence. We identify and analyze users that participate in multiple online communities, look at some of the high-priced zero-day exploits for sale, discuss how governmentassigned vulnerability identifiers are used to indicate a product's target, and use unsupervised learning to categorize and study the product offerings of 17 darknet marketplaces. For product categorization, we use a combination of manual labeling with clustering techniques to identify specific categories. Through a series of case studies showcasing various findings relating to malicious hacker behavior, we hope to illustrate the utility of these cyber threat intelligence tools.
The price of a given product on a darknet marketplace is typically indicated in Bitcoin. The BTC to USD conversion rate is highly volatile. At the time of writing, the Bitcoin to USD conversion rate was $649.70 to 1 BTC, whereas during the experiments discussed during this chapter, which occurred only a few months prior to the writing of this book, the conversion rate was $380.03 to 1 BTC.
The goal of a cyber threat intelligence system is to aid cybersecurity professionals with their strategic cyber-defense planning and to address questions such as:
1 What vendors and users have a presence in multiple darknet/deepnet markets/forums?
2 What zero-day exploits are being developed by malicious hackers?
3 What vulnerabilities do the latest exploits target?
4 What types of products are exclusive to certain vendors and markets?
After aggregating the hacking-related products and hacking-related discussions from a number of darknet marketplaces and forums, respectively, we can begin answering these questions via an in-depth analysis of the data in order to provide a better understanding of the interactions within and between these communities.
Marketplace Data Characteristics
In this section, we describe the dataset used in this chapter. We examined the hacking-related products from 17 darknet marketplaces, finding many products that were cross-posted between markets, often by vendors of the same username. Figure 5.1 shows the count of vendors using the same screen-name across multiple marketplaces and Table 5.1 displays the dataset statistics, after removing duplicates (cross-posts).
7 - Application: Protecting Industrial Control Systems
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 96-117
-
- Chapter
- Export citation
-
Summary
Introduction
In the last chapter, we explored how to determine a cyber-attacker's optimal strategy for attacking a computer system based on malware and exploits available on the darkweb. In this chapter, we look at the case where the attacker is focused on industrial control systems (ICS): IT infrastructure that controls physical systems (electricity, water, industrial machinery, etc.). A critical feature of these complex ICS systems is the interdependencies among various components.
However, despite the prevalence of markets for malware and exploits, and their potential threat to ICS, existing paradigms, including the framework presented in the previous chapter, do not account for the complex nature of ICS systems consisting of multiple interconnected components. In particular, it would prove useful to simulate a cyber-attack on a model of an existing system, to assess its degree of vulnerability. Such a model would also prove useful for automated cybersecurity systems that can learn defense and contingency strategies based on the model's simulations. This chapter takes the first steps toward addressing this need. In particular, we introduce a framework that allows for modeling of ICS systems with highly interconnected components (Section 7.3) and study this model through the lens of lattice theory [57]. We then turn our attention to the problem of determining the optimal/most dangerous strategy for a cyber-adversary with respect to this model and find it to be an NPComplete problem (Section 7.4). Next, we present a suite of algorithms for this problem based on A* search and introduce provably correct algorithms (Section 7.5). Our intuition is that these algorithms will obtain satisfactory performance in practice due to heuristic functions (for which we show admissibility). We demonstrate the performance of these algorithms by implementing them and performing a suite of experiments using both simulated and actual vulnerability data (Section 7.6). This chapter also includes some background on ICS (Section 7.2) and a brief overview of related work (Section 7.7).
Background
Contemporary cyber threat actors rely on a variety of malware and exploits purchased through various channels such as the darkweb [99] in order to carry out their attacks. The trend toward automation of industrial control systems (ICS) and toward “smart” utilities [50] has made understanding such adversarialbehavior directed against ICS a priority. For instance, code from the infamous Stuxnet [97] attack against Iranian nuclear facilities is available for public download.
Darkweb Cyber Threat Intelligence Mining
- John Robertson, Ahmad Diab, Ericsson Marin, Eric Nunes, Vivin Paliath, Jana Shakarian, Paulo Shakarian
-
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017
-
The important and rapidly emerging new field known as 'cyber threat intelligence' explores the paradigm that defenders of computer networks gain a better understanding of their adversaries by understanding what assets they have available for an attack. In this book, a team of experts examines a new type of cyber threat intelligence from the heart of the malicious hacking underworld - the dark web. These highly secure sites have allowed anonymous communities of malicious hackers to exchange ideas and techniques, and to buy/sell malware and exploits. Aimed at both cybersecurity practitioners and researchers, this book represents a first step toward a better understanding of malicious hacking communities on the dark web and what to do about them. The authors examine real-world darkweb data through a combination of human and automated techniques to gain insight into these communities, describing both methodology and results.
2 - Moving to Proactive Cyber Threat Intelligence
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 4-12
-
- Chapter
- Export citation
-
Summary
Introduction
Cybersecurity is often referred to as offense dominant,meaning that the domain generally favors the attacker [67, 65]. The reasoning behind this is simple: a successful defense must block all pathways to a system while a successful attack requires only one. As the old hacker adage goes: “the defender must always be right—the attacker only needs to be right once.” This notion of an offense dominant cybersecurity stems directly from “best practices” in the field. These methods primarily rely on technical measures to improve defense. Traditionally these have included variations on patch management, firewall usage, intrusion detection, and antivirus. However, an adversary particularly keen on gaining access to a system can study such defenses with the goal of finding the gaps. These actions are not limited to nation states or large criminal enterprises. The community of malicious hackers is a key enabler for these activities. While important, technical defense measures alone are unlikely to halt attackers and the offense will have the advantage in this case. This chapter explores the use of cyber threat intelligence to address this problem. By gaining insights on the adversary's behavior, we can better address the offense-dominant problem inherent in cybersecurity. The new market for cyber threat intelligence has emerged in recent years due to the realization that technical defensivemeasures, by themselves, are insufficient to address cybersecurity.
Consider the Threat
Central to the idea of cyber threat intelligence (in its current incarnation) is the sharing of information on the latest observed threats. Such data may be collected by a third party (i.e., a company that specializes in incident response or network monitoring), shared directly between organizations, or shared through a group of organizations (i.e., the various Information Sharing and Analysis Centers or ISACs). Certainly, distribution in a manner that best maximizes such information sharing while respecting the privacy of organizations and individuals is a key concern here, as is the role of government in such arrangements. These are some of several short-term problems that are being addressed by threat intelligence firms today: big data management; identification of attack patterns; sanitization/dissemination of information; knowledge extraction; and others. However, these are all relatively short-term problems. This chapter focuses on a larger, more systemic issue with cyber threat intelligence as it stands today: the vast majority of it is inherently reactive.
Glossary
- John Robertson, Arizona State University, Ahmad Diab, Arizona State University, Ericsson Marin, Arizona State University, Eric Nunes, Arizona State University, Vivin Paliath, Arizona State University, Jana Shakarian, Arizona State University, Paulo Shakarian, Arizona State University
-
- Book:
- Darkweb Cyber Threat Intelligence Mining
- Published online:
- 06 April 2017
- Print publication:
- 04 April 2017, pp 123-126
-
- Chapter
- Export citation